arq: MFSK weak-signal backend + delivery-driven stop-and-wait rethink - #125

The real question for a weak-signal mode is acquisition, not just modulation.
Record what is confidently known: v1's MFSK carries its own non-coherent
preamble detector (not OFDM coherent sync), and the codec2 FSK_LDPC end-to-end
figures (acquisition included) already beat DATAC15 at the fringe — so the
non-coherent-FSK class is not acquisition-limited the way OFDM is. Also record
the limitation: an isolated MFSK acquisition sim confirmed detector mechanics
but could not be calibrated to the SNR3k axis; a trustworthy MFSK-specific floor
needs end-to-end integration (framing + time_sync_mfsk_corr + LDPC in ch), the
next step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The MFSK-relevant slice of v1's cl_ofdm: self-contained radix-2 FFT/IFFT (no
FFT-library dependency), subcarrier zero-pad/de-pad (DC-centred v1 layout), and
cyclic-prefix add/remove. fft is 1/N-normalized, ifft unnormalized (v1
convention) so the pair is identity. This carries mfsk.c's tone bins to/from
OFDM time-domain symbols — the framing layer needed to run the MFSK mode through
a real channel end-to-end. Validated by a round-trip unit test (bins -> pad ->
ifft -> +CP -> -CP -> fft -> depad recovers bins at machine precision).

Next: port time_sync_mfsk_corr (non-coherent acquisition) + baseband channel +
LDPC to measure the MFSK acquire+decode floor vs DATAC15.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Pure-C port of v1 cl_ofdm::time_sync_mfsk_corr: non-coherent (envelope)
matched-filter preamble detection — two-phase search (coarse 4x-oversampled +
fine), per-symbol normalized correlation averaged over the preamble symbols,
per-symbol floor 0.05, threshold 0.5, earliest-strong-preamble early exit.
Needs no phase/frequency lock, so it acquires below the coherent-OFDM floor —
this is the acquisition path the MFSK weak-signal mode uses (not OFDM coherent
sync). mfsk_sync_build_template() builds the preamble time template via the
mfsk + ofdm framing. Unit test plants the template in a noise buffer: detected
at the right offset (high SNR), rejected on pure noise.

With mfsk.c (codec) + mfsk_ofdm.c (framing) + mfsk_sync.c (acquisition), the
MFSK mode's TX/RX signal-processing core is now in C. Remaining for an
SNR3k-comparable end-to-end floor vs DATAC15: baseband<->passband + carrier +
interpolation chain through ch (large; the SNR3k calibration lives there).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…s the concern)

Wired the ported MFSK core (mfsk.c + mfsk_ofdm.c + mfsk_sync.c) into a real
passband pipeline through codec2 `ch` (SNR3k measured exactly as for DATAC15).
Result: v1's non-coherent MFSK acquisition holds (metric 0.6-0.9) far below
DATAC15's ~-7 dB acquisition wall — to ~-13 dB SNR3k on AWGN (matching v1's
claim) and past -11 dB on MPP fading (uncoded BER <=3%, where DATAC15 delivers
22%), before any LDPC. Acquisition is NOT the bottleneck; the mode carries its
own. Replaces the earlier "couldn't measure trustworthily" caveat with the
ch-calibrated numbers. Remaining work is wiring it as an actual ARQ-ladder mode
(LDPC + mode-pool + OLLA), not whether it can acquire.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…floor)

Port v1's ROBUST_0 FEC — rate-1/16 LDPC (N=1600, K=100): systematic IRA encoder
(mfsk_ldpc_encode) + normalized min-sum BP decoder (mfsk_ldpc_decode) over the v1
quasi-cyclic Tanner graph, matrix embedded in mfsk_ldpc_1_16.c. Verified:
encoder produces valid codewords (H*c=0), noiseless encode->decode lossless
(unit test). Wired end-to-end through the passband/ch pipeline (TX info->encode
->32-MFSK->passband->ch->RX acquire->demod->LDPC decode).

Coded floor (15 frames/point): delivered == acquired everywhere — once acquired
the code always decodes, so the mode is acquisition-limited, not decode-limited.
Coded MFSK delivers ~2-3 dB deeper than DATAC15 on fading (47% at -10.8 dB SNR3k
vs DATAC15 22%; 33% at -12.8 where DATAC15 ~0). Next lever is acquisition
(postamble / longer preamble), not the code. See docs/MFSK-PORT.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ison

Since the coded MFSK mode is acquisition-limited, add a postamble: a second
known-tone sequence (distinct tones, +2 offset) after the payload, so RX syncs
on preamble OR postamble. mfsk_generate_postamble + mfsk_sync_build_postamble_
template; harness does dual-ended search. Measured (MPP, coded, same rx file):
dual-ended buys ~2 dB — 87% vs 60% delivery at -10.8 dB SNR3k, 100% vs 80% at
-8.8.

Full mode comparison (MPP, delivered %): dual-ended coded MFSK 100/87/67/20 at
SNR3k -8.8/-10.8/-12.8/-14.8 vs DATAC16 63/40/20, DATAC15 50/33/3, DATAC1/DATAC3
0. The MFSK weak-signal mode extends the floor ~3-4 dB below the most robust
OFDM modes, at ~8 bps (deep-fringe bottom rung). Unit test added for the
postamble. See docs/MFSK-PORT.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… was wrong

Generalise mfsk_ldpc to a code-descriptor + registry and port v1's full rate
ladder: 1/16, 2/16, 3/16, 5/16, 8/16 (rate 1/2), all N=1600 (same airtime),
payload K/8 = 12.5..100 bytes. Generic systematic IRA encoder + min-sum decoder
take a mfsk_ldpc_code_t; unit test validates encode->H*c=0->noiseless-decode for
all five rates.

Measured (32-MFSK, dual-ended, MPP through ch): all rates deliver identically
(delivered==acquired; even rate 1/2 decodes everything acquired down to the
acquisition floor, since the 32-MFSK demod hands the code very clean LLRs). So
the LDPC rate barely affects the floor — it only sets payload. 1/16 (12.5 B)
was the wrong default: it can't even fit the 14-byte CONNECT frame, whereas
8/16 (100 B, ~60 bps, ~DATAC15 throughput at ~3-4 dB more robustness) does.
Recommend shipping a small ladder (5/16, 8/16) for OLLA. Caveat: "rate is free"
holds in idealised sim (no freq offset / drift over the ~13 s frame); real HF
may favour shorter frames — set the ceiling OTA. See docs/MFSK-PORT.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Port cl_ofdm::detect_ack_pattern (Phase-1 peak-bin match count) to pure C as
mfsk_detect_pattern in mfsk_sync.c, over the depadded-bins framing (same path
as mfsk_demod) rather than v1's raw-FFT-bin + carrier-image handling, which
this pipeline's LPF+depad already covers.

Measured pattern-ACK vs a codec2 DATAC16 ACK through ch/Watterson: an ACK burst
is 0.64s (vs 3.74s DATAC16, 5.8x less airtime) and survives ~10-12 dB deeper on
fading -- 100% to ~-9 dB / >75% to ~-13 dB on moderate+poor, where DATAC16 is
dead by ~-5 dB. Pure-noise false-alarm gate: 0/30 down to -26 dB. This directly
addresses the reverse-path ACK-survival blocker; confirms v1's split of coded
MFSK frames (data + CONNECT) vs Welch-Costas patterns (ACK/BREAK/keepalive).

Unit test: planted ACK detected, noise rejected, BREAK not confused for ACK.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Mercury's datalink is codec-agnostic (opaque frame bytes + CRC in/out), but the
modem layer historically called codec2/FreeDV directly. Introduce a
modem_backend_t vtable (modem_backend.h) so more than one modem architecture can
coexist behind one seam -- the original mercuryv2 design intent. FreeDV becomes
a first-class backend (modem_freedv.c), a 1:1 adapter over the freedv_api.

- generic_modem_t and rx_decoder_state_t now hold a modem_codec_t {backend, ctx}
  instead of a struct freedv *.
- The per-mode pool is a generic slot array keyed by mode (was a hand-enumerated
  freedv-only struct); clear/open/lookup dispatch through the backend.
- Both DSP funnels (send_modulated_data TX, rx_decoder_consume_chunk RX) plus
  bind/geometry/bitrate/spectrum route through the vtable; no freedv special-case.
- CRC16 (freedv_gen_crc16) stays codec-independent in the TX funnel.
- mod_out_short sized to max(pre/frame/postamble) so a burst backend can't
  overflow it.

No behaviour change: full unit suite green; TX output byte-identical to the
former direct-freedv path across DATAC15/16/1/3/4/17/QAM16C2 (parity harness).
This is the foundation for the MFSK weak-signal backend (Stage 2).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add modem_mfsk.c: the non-coherent 32-MFSK burst codec behind the
modem_backend_t vtable, mode MERCURY_MODE_MFSK (100, clear of the freedv enum).

- TX: preamble_tx/rawdata_tx/postamble_tx build baseband OFDM (rate-1/2 LDPC,
  100-byte frame) and mix to the 2 kHz passband as int16, phase-continuous
  across the burst. Same frame contract as freedv (98 payload + 2 CRC16).
- RX: keeps its own sliding sample window and presents the freedv-style
  nin/rawdata_rx interface -- accumulates symbols, downmix+LPF, non-coherent
  preamble correlation (mfsk_sync_search), energy demod, LDPC decode, and
  returns bytes ONLY on CRC16-valid (freedv's contract). Detection is rate-
  limited to a few attempts per burst.
- backend_for_mode() routes MERCURY_MODE_MFSK to it; the mfsk sources are now
  linked into the mercury binary and the UI core lib.

Not yet a selectable mode (no pool/ladder entry) so default behaviour is
unchanged -- that is Stage 3. Deterministic round-trip unit test: TX a frame ->
passband -> feed the RX in nin-sized chunks -> exact bytes back, CRC-gated;
pure noise never false-decodes. Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Make MERCURY_MODE_MFSK a selectable payload mode reached only when the OLLA-
corrected SNR drops below the DATAC15 floor (~-11 dB) -- deep fade only, so
default behaviour is unchanged.

modem.c: MFSK is now a supported + payload split mode (RX payload decoder binds
to it when the peer TXes MFSK), registered in the pool, named, and given a
bitrate level.

arq: arq_mode_table gets the MFSK row (unique 98-byte payload so DATA-frame mode
inference stays unambiguous; ~13.5s frame, generous ack/retry). New
ARQ_SNR_MIN_DATAC15_DB (-11) sets the DATAC15->MFSK drop point; select_best_mode
terminal now returns DATAC15 above it and MFSK below, with the standard +5 dB
upgrade hysteresis so the link won't oscillate into the long MFSK frame.
mode_rank(MFSK) = -1 (below DATAC15); MFSK entry is OLLA-driven, not retry-count
(the hard-loss floor stays DATAC15). Session still starts at DATAC15.

Deterministic test (test_arq_olla): DATAC15 held to -11, drops to MFSK at -12,
and climbs back only above -6 (threshold+hysteresis). Full suite green; binary
builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Replace the SNR/OLLA-negotiated, go-back-N, restage-buffered data-flow FSM with
a simple delivery-driven stop-and-wait protocol that starts at the most robust
rung (MFSK) and climbs. arq_fsm.c drops from 2466 to ~1470 LOC.

- Data-flow FSM: 15 -> 5 states (IDLE_ISS/DATA_TX/WAIT_ACK/IDLE_IRS/ACK_TX);
  connection FSM (connect/disconnect handshake) unchanged. 21 -> 11 states.
- Control ACK is now a Welch-Costas PATTERN (0.64s, ~10dB more robust than the
  3.74s DATAC16 ACK it replaces) with a 2-symbol alphabet: ACK / ACK+TURN
  (== HAS_DATA piggyback). Emitted via ARQ_ACTION_TX_PATTERN + send_pattern_ack
  in modem.c; detected by a third RX consumer (mfsk_pattern_detect) that
  synthesizes ARQ_EV_RX_ACK. mfsk_pattern_tx/detect added to modem_mfsk.
- Mode control: delivery-driven ladder MFSK->DATAC15->..->QAM16C2 indexed by
  speed_level (fast initial ramp, then N-clean-per-step; any retry steps down).
  NO SNR, NO OLLA offset, NO reverse-hold gate. Host SNR display is unaffected
  (it comes from decoding received DATA frames, not the ACK).
- TX buffering: one immutable retained frame (raw user bytes, fixed seq<->content)
  instead of the 5-slot window + restage. A mode drop re-frames the same bytes at
  the smallest mode that still fits (mode_that_fits) -- never resized, so a
  duplicate is idempotent on the peer.
- Deleted: MODE_REQ/ACK + TURN_REQ/ACK + KEEPALIVE states/handlers/builders,
  select_best_mode/maybe_upgrade_mode/OLLA/restage; wire flags TURN_REQ/
  CTRL_ACKSEQ/BURST_END; SNR/OLLA/hard-loss/mode-hold constants. IRS liveness now
  via the no-progress net (replaces keepalive). Turn handoff is piggyback-only,
  with a CALLER-keeps-floor role tiebreak on simultaneous bids.

Asymmetric links: each station adapts its own forward mode from its own delivery
outcomes; the robust pattern ACK survives the weak reverse path, so the healthy
forward mode is not spuriously downgraded (the reason the old reverse-hold gate
existed) -- validated by test_sim_asymmetric_strong_forward.

Tests: full suite green (172 tests). test_arq_fsm rewritten for the 5-state flow;
test_arq_olla deleted -> test_arq_ladder (deterministic climb/drop, MFSK start);
new test_pattern_ack_detection (DSP: 0 false ACKs on noise, ack/break discrimination);
test_arq_sim gains pattern roundtrip, fade->MFSK+recover with byte-integrity,
bidirectional piggyback, lost-ACK idempotency, asymmetric strong/weak. Sim models
the pattern ACK as a short-airtime outframe with per-direction channel SNR.

Bugs the sim caught and fixed in logic (not hidden): mode-drop duplicate-byte
corruption (immutable frame), bidirectional double-ISS deadlock (role tiebreak),
IRS-never-disconnects (inactivity net). Flat-erasure fuzz ceiling lowered 0.40->
0.25: at flat high erasure 'drop-on-retry' crawls (no stall/corruption -- probed
to completion); the accepted slower-at-high-loss tradeoff, tuned at OTA.

OTA-gated (Pedro, Sao Roque <-> Belo Horizonte) before merge to mercuryv2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The ARQ rethink passed the deterministic sims but failed the Go integration
harness (TestMercuryARQTransfer, real codec path over the ch bridge): the
session never connected, and once connect was fixed the MFSK-floor data
delivered 0 bytes. Two independent root causes, both in the live RX loop that
the abstract sims don't exercise:

1) Connect handshake starved by the pattern-ACK detector.
   The new Welch-Costas pattern detector (3rd RX consumer) ran its sliding-
   window correlation on *every* chunk, gated on arq_policy_ready — which is
   always true once the engine is up. That per-chunk cost slowed the RX loop
   enough to miss the DATAC16 ACCEPT on the timing-marginal CALL/ACCEPT
   turnaround, so the caller retried CALL forever and never connected. A
   pattern ACK can only arrive in two states (answerer awaiting the connect-
   confirm = ACCEPTING, or a live session awaiting a data ACK = CONNECTED), so
   gate the detector on a new snapshot flag expect_pattern_ack and skip it
   during CALLING/LISTENING/idle. (mercuryv2 has no such consumer and connects
   fine — this was a pure regression of the rethink.)

2) MFSK data burst chopped by the RX backlog flush.
   The capture-backlog flush used a fixed 2 s cap (sized for fast FreeDV
   frames). An MFSK payload frame is a single ~13.5 s burst; the 2 s cap
   flushed it ~6x mid-burst, so the MFSK sliding-window sync never saw a
   contiguous burst and decoded 0. Size the cap to the active payload frame
   duration + 3 s guard so an in-flight burst is never dropped, while fast
   modes keep the tight 2 s latency bound (preserves the issue 81 protection).

With both fixes the harness passes end-to-end starting at the MFSK floor
(full 102 B delivered, 0 backlog flushes); DATAC15-start also passes. Full
unit+sim suite green; backtoback/control/bidir/large_queue integration green.

Debugging note: mercury's own -v (DEBUG logging) is heavy enough to perturb
the marginal connect timing (a Heisenbug) — validate with exit codes + a
single promoted INFO log, not full -v.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ean checkout

test_mfsk_modem links ../modem/freedv/libfreedvdata.a, but that library is
produced by the freedv sub-build, not by tests/Makefile — so a clean checkout
(CI's UT job runs `make test` with no prior build) failed to link it
("ld returned 1 exit status"). This predates the connect/MFSK fixes; it only
passed locally because a prior full build had left the .a in place.

Add libfreedvdata.a as a prerequisite of test_mfsk_modem with a rule that runs
the freedv sub-build, so `make test` is self-contained from clean. Verified:
rm the .a + full `make -C tests test` from clean → all tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Follow-up to the dialed-SSID callback change (notify_connected/notify_
pending now carry the local call).  test_arq_ladder.c is created on this
branch by the data-plane rewrite, so the mercuryv2 SSID fix could not
update its FFF fakes during the rebase; bring them to the 2-arg signature
so the branch builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Rebased mfsk-arq-integration onto mercuryv2 which now carries PR #127's HARQ
combined-decode parity gate (freedv_700.c + mpdecode_core.h come in via the
base cleanly). Re-add PR #127's test_freedv_harq to this branch's (divergent)
tests/Makefile — TEST_BINS entry + build rule — which the rebase dropped while
resolving the Makefile toward each MFSK commit. Full suite green incl.
test_freedv_harq 3/3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The MFSK RX synced only on the preamble at the burst head, so a burst
whose head was clipped -- the fragile part of a half-duplex burst (far end
still keyed, PTT releasing, AGC/T-R turnaround settling) -- was
unrecoverable and cost a full ACK-timeout retransmit. Over the -x sock
virtual clock this stalled the transfer: the ISS starts its first ~13.5 s
data burst while the IRS is finishing its connect turnaround, so the IRS
drains the burst's preamble and never decodes it (observed: burst 1
best_metric=0.078 NO preamble; a later clean retransmit metric=0.885).

The postamble is the same P known symbols emitted after the data; its
template was already built (h->pstT/pstE/pstN) but unused. When the
preamble path fails, search for the postamble and anchor the payload from
the tail (payoff = poff - NPAY*Nofdm). Recovers head-clipped bursts; a
general HF robustness win, not just for -x sock.

Test: test_mfsk_modem_preamble_clipped_recovers_via_postamble drops the
preamble entirely and requires the exact frame back.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The IRS payload decoder tracked peer_tx_mode = the last-DECODED mode
(arq_modem_preferred_rx_mode() is hardcoded to DATAC16, so
select_payload_rx_mode's first branch is dead). When the ISS climbed the
ladder (e.g. MFSK->DATAC15) the IRS's dual decoder had no slot for the new
mode, missed the first burst of every climb, never ACKed it, and the ISS
retried and fell back -- the transfer oscillated at the MFSK floor and
crawled.

The IRS observes the same per-frame outcomes the sender climbs on, so it
now mirrors the same delivery-driven ladder: a clean new frame climbs, a
duplicate (sender retried and stepped down) steps down, and a full
IDLE_IRS idle-hold with no RX steps down toward the floor (reset-on-miss,
so a lost ACK that left us climbed above the sender re-rendezvous at the
floor). Keeps the payload decoder on the mode the peer's NEXT burst will
use, with no on-wire mode negotiation. ladder_step() is factored out of
record_tx_outcome so both ends apply identical rules.

Tests: test_irs_mirror_climbs_with_peer / _steps_down_on_duplicate /
_resets_toward_floor_on_silence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The data-plane rethink rewrote arq_fsm.c against a base that predates the
LISTEN OFF fix (#141), so rebasing onto mercuryv2 would silently drop it and
merging this branch would reintroduce the BPQ32 interlock bug.

Re-applied against the new state machine:

  - CALLING / ACCEPTING: fall through to the APP_DISCONNECT teardown.
  - CONNECTED: abort semantics — no backlog drain, no air-side DISCONNECT
    frame; the peer times out. Deliberately unlike APP_DISCONNECT, which
    defers to drain.
  - DISCONNECTING: stop retransmitting DISCONNECT.

Same three tests as on trunk, rewritten against the new fixtures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>